Systems and methods for processing objects, including automated processing

ABSTRACT

A processing system for processing objects using a programmable motion device is disclosed. The processing system includes a plurality of supply bins providing supply of a plurality of objects, with the plurality of supply bins being provided with a bin conveyance system, a programmable motion device in communication with the bin conveyance system, where the programmable motion device includes an end effector for grasping and moving a selected object out of a selected supply bin, and a movable carriage for receiving the selected object from the end effector of the programmable motion device, and for carrying the selected object to one of a plurality of destination containers.

PRIORITY

The present application claims priority to U.S. Provisional PatentApplication Ser. No. 62/476,310 filed Mar. 24, 2017, the disclosure ofwhich is hereby incorporated by reference in its entirety.

BACKGROUND

The invention generally relates to automated programmable motion controlsystems, e.g., robotic, sortation and other processing systems, andrelates in particular to programmable motion control systems intendedfor use in environments requiring that a variety of objects (e.g.,articles, packages, consumer products etc.) be processed and moved to anumber of processing destinations.

Many object distribution systems, for example, receive objects in adisorganized stream or bulk transfer that may be provided as individualobjects or objects aggregated in groups such as in bags, arriving on anyof several different conveyances, commonly a conveyor, a truck, a palleta Gaylord, or a bin etc. Each object must then be distributed to thecorrect destination location (e.g., a container) as determined byidentification information associated with the object, which is commonlydetermined by a label printed on the object. The destination locationmay take many forms, such as a bag, a shelf, a container, or a bin.

The processing (e.g., sortation or distribution) of such objects hastraditionally been done, at least in part, by human workers that scanthe objects, for example with a hand-held barcode scanner, and thenplace the objects at assigned locations. Many order fulfillmentoperations, for example, achieve high efficiency by employing a processcalled wave picking. In wave picking, orders are picked from warehouseshelves and placed at locations (e.g., into bins) containing multipleorders that are sorted downstream. At the sorting stage, individualarticles are identified, and multi-article orders are consolidated, forexample, into a single bin or shelf location, so that they may be packedand then shipped to customers. The process of sorting these articles hastraditionally been done by hand. A human sorter picks an article, andthen places the article in the so-determined bin or shelf location whereall articles for that order or manifest have been defined to belong.Automated systems for order fulfillment have also been proposed. See,for example, U.S. Patent Application Publication No. 2014/0244026, whichdiscloses the use of a robotic arm together with an arcuate structurethat is movable to within reach of the robotic arm.

The identification of objects by code scanning generally either requiremanual processing, or require that the code location be controlled orconstrained so that a fixed or robot-held code scanner (e.g., a barcodescanner) can reliably detect the code. Manually operated barcodescanners are therefore generally either fixed or handheld systems. Withfixed systems, such as those at point-of-sale systems, the operatorholds the article and places it in front of the scanner, which scanscontinuously, and decodes any barcodes that it can detect. If thearticle's code is not immediately detected, the person holding thearticle typically needs to vary the position or orientation of thearticle with respect to the fixed scanner, so as to render the barcodemore visible to the scanner. For handheld systems, the person operatingthe scanner may look at the barcode on the article, and then hold thearticle such that the barcode is within the viewing range of thescanner, and then press a button on the handheld scanner to initiate ascan of the barcode.

Further, many distribution center sorting systems generally assume aninflexible sequence of operation whereby a disorganized stream of inputobjects is provided (by a human) as a singulated stream of objects thatare oriented with respect to a scanner that identifies the objects. Aninduction element or elements (e.g., a conveyor, a tilt tray, ormanually movable bins) transport the objects to desired destinationlocations or further processing stations, which may be a bin, a chute, abag or a conveyor etc.

In conventional object sortation or distribution systems, human workersor automated systems typically retrieve object s in an arrival order,and sort each object or object into a collection bin based on a set ofgiven heuristics. For example, all objects of a like type might bedirected to a particular collection bin, or all objects in a singlecustomer order, or all objects destined for the same shippingdestination, etc. may be directed to a common destination location.Generally, the human workers, with the possible limited assistance ofautomated systems, are required to receive objects and to move each totheir assigned collection bin. If the number of different types of input(received) objects is large, then a large number of collection bins isrequired.

FIG. 1 for example, shows an object distribution system 10 in whichobjects arrive, e.g., in trucks, as shown at 12, are separated andstored in packages that each include a specific combination of objectsas shown at 14, and the packages are then shipped as shown at 16 todifferent retail stores, providing that each retail store receives aspecific combination of objects in each package. Each package receivedat a retail store from transport 16, is broken apart at the store andsuch packages are generally referred to as break-packs. In particular,incoming trucks 12 contain vendor cases 18 of homogenous sets ofobjects. Each vendor case, for example, may be provided by amanufacturer of each of the objects. The objects from the vendor cases18 are moved into decanted bins 20, and are then brought to a processingarea 14 that includes break-pack store packages 22. At the processingarea 14, the break-pack store packages 22 are filled by human workersthat select items from the decanted vendor bins to fill the break-packstore packages according to a manifest. For example, a first set of thebreak-pack store packages may go to a first store (as shown at 24), anda second set of break-pack store packages may go to a second store (asshown at 26). In this way, the system may accept large volumes ofproduct from a manufacturer, and then re-package the objects intobreak-packs to be provided to retail stores at which a wide variety ofobjects are to be provided in a specific controlled distributionfashion.

Such a system however, has inherent inefficiencies as well asinflexibilities since the desired goal is to match incoming objects toassigned collection bins. Such systems may require a large number ofcollection bins (and therefore a large amount of physical space, largeinvestment costs, and large operating costs), in part, because sortingall objects to all destinations at once is not always most efficient.Additionally, such break-pack systems must also monitor the volume ofeach like object in a bin, requiring that a human worker continuouslycount the items in a bin.

Further, current state-of-the-art sortation systems also rely in humanlabor to some extent. Most solutions rely on a worker that is performingsortation, by scanning each object from an induction area (chute, table,etc.) and placing each object at a staging location, conveyor, orcollection bin. When a bin is full, another worker empties the bin intoa bag, box, or other container, and sends that container on to the nextprocessing step. Such a system has limits on throughput (i.e., how fastcan human workers sort to or empty bins in this fashion) and on numberof diverts (i.e., for a given bin size, only so many bins may bearranged to be within efficient reach of human workers).

Unfortunately, these systems do not address the limitations of the totalnumber of system bins. The system is simply diverting an equal share ofthe total objects to each parallel manual cell. Thus, each parallelsortation cell must have all the same collection bin designations;otherwise, an object may be delivered to a cell that does not have a binto which the object is mapped. There remains a need, therefore, for amore efficient and more cost effective object processing system thatprocesses objects of a variety of sizes and weights into appropriatecollection bins or trays of fixed sizes, yet is efficient in handlingobjects of varying sizes and weights.

SUMMARY

In accordance with an embodiment, the invention provides a processingsystem for processing objects using a programmable motion device. Theprocessing system includes a plurality of supply bins providing supplyof a plurality of objects, with the plurality of supply bins beingprovided with a bin conveyance system, a programmable motion device incommunication with the bin conveyance system, where the programmablemotion device includes an end effector for grasping and moving aselected object out of a selected supply bin, and a movable carriage forreceiving the selected object from the end effector of the programmablemotion device, and for carrying the selected object to one of aplurality of destination containers.

In accordance with another embodiment, the invention provides aprocessing system for processing objects using a programmable motiondevice, where the processing system includes a programmable motiondevice including an end effector for grasping and moving a selectedobject from a plurality of objects, a movable carriage for receiving theselected object from the end effector of the programmable motion device,and for carrying the selected object to one of a plurality ofdestination containers, and a destination container displacement systemfor urging a selected destination container onto an output conveyor.

In accordance with a further embodiment, the invention provides a methodof processing of objects. The method includes the steps of providing aplurality of supply bins including a plurality of objects, with theplurality of supply bins being provided with a bin conveyance system,grasping and moving a selected object out of a selected supply bin usinga programmable motion device, providing the selected object to areciprocating carriage, and carrying the selected object to one of aplurality of destination containers using the reciprocating carriage.

BRIEF DESCRIPTION OF THE DRAWINGS

The following description may be further understood with reference tothe accompanying drawings in which:

FIG. 1 shows an illustrative diagrammatic view of an object processingsystem of the prior art;

FIG. 2 shows an illustrative diagrammatic view of an object processingsystem in accordance with an embodiment of the present invention;

FIG. 3 shows an illustrative diagrammatic view of a processing sectionof the object processing system of FIG. 2;

FIG. 4 shows an illustrative diagrammatic view of an object processingsystem in accordance with another embodiment of the present invention;

FIG. 5 shows an illustrative diagrammatic view of the perception systemof FIGS. 2-4;

FIG. 6 shows an illustrative diagrammatic view from the perceptionsystem of FIGS. 2-4, showing a view of objects within a bin of objectsto be processed;

FIGS. 7A and 7B show an illustrative diagrammatic view of a graspselection process in an object processing system of an embodiment of thepresent invention;

FIGS. 8A and 8B show an illustrative diagrammatic view of a graspplanning process in an object processing system of an embodiment of thepresent invention;

FIGS. 9A and 9B show an illustrative diagrammatic view of a graspexecution process in an object processing system of an embodiment of thepresent invention;

FIG. 10 shows an illustrative diagrammatic top view of an objectprocessing system in accordance with another embodiment of the inventionthat identifies changing and unchanging motion planning general areas;

FIG. 11 shows an illustrative diagrammatic top view of the system ofFIG. 10, showing multiple possible paths from the programmable motiondevice to a destination carrier;

FIG. 12 shows an illustrative diagrammatic top view of the system ofFIG. 10, showing a path from the programmable motion device to adestination carrier with an emphasis on minimum time;

FIG. 13 shows an illustrative diagrammatic top view of the system ofFIG. 10, showing a path from the programmable motion device to adestination carrier with an emphasis on minimum risk;

FIG. 14 shows an illustrative diagrammatic view of a processing sectionin an object processing system in accordance with an embodiment of theinvention wherein an object is placed in a carriage;

FIG. 15 shows an illustrative diagrammatic view of the processingsection of FIG. 14 with the carriage having been moved along its track;

FIG. 16 shows an illustrative diagrammatic view of the processingsection of FIG. 14 with the carriage having transferred its load to adestination bin;

FIGS. 17A and 17B show illustrative diagrammatic views of a bin removalmechanism for use in an object processing system in accordance with anembodiment of the invention;

FIG. 18 shows an illustrative diagrammatic view of the processingsection of FIG. 14 with the carriage having returned to its base, and aremoved destination bin being moved urged from its location;

FIG. 19 shows an illustrative diagrammatic view of the processingsection of FIG. 14 with the removed destination bin being moved along anoutput conveyor;

FIG. 20 shows an illustrative diagrammatic exploded view of a boxassembly for use as a storage bin or destination bin in accordance withvarious embodiments of the present invention;

FIG. 21 shows an illustrative diagrammatic view of the assembled boxtray assembly of FIG. 20;

FIG. 22A-22D show illustrative diagrammatic views of a furtherembodiment of a bin displacement system for use in further embodimentsof the invention;

FIG. 23 shows an illustrative diagrammatic view of an object processingsystem in accordance with a further embodiment of the present inventionemploying a plurality of storage sections, processing sections, and aplurality of destination sections; and

FIG. 24 shows an illustrative diagrammatic view of an object processingsystem in accordance with a further embodiment of the present inventionemploying further pluralities of storage sections, processing sectionsand destination sections.

The drawings are shown for illustrative purposes only.

DETAILED DESCRIPTION

In accordance with an embodiment, the invention provides a processingsystem for processing objects using a programmable motion device. Theprocessing system includes a plurality of supply bins, a programmablemotion device, and a movable carriage. The plurality of supply binsprovide supply of a plurality of objects, and the plurality of supplybins are provided with a bin conveyance system. The programmable motiondevice is in communication with the bin conveyance system, and theprogrammable motion device includes an end effector for grasping andmoving a selected object out of a selected supply bin. The movablecarriage is for receiving the selected object from the end effector ofthe programmable motion device, and is used for carrying the selectedobject to one of a plurality of destination containers.

With reference to FIG. 2, a system 30 in accordance with an embodimentof the present invention includes an input conveyor 34 on which supplybins 32 (e.g., decanted vendor bins) are provided, a processing section36 that includes a programmable motion device 40, and a destinationsection 44 that includes one or more shuttle carriers 42. Generally,supply bins 32 are selectively provided to and from the processingsection 36 by a diverter conveyor 35 that moves bins of objects to theprocessing section 36. The programmable motion device 40 then movesobjects from a selected supply bin 32 and places or drops them into acarriage 42 of the destination section 44 for delivery to one of aplurality of destination containers 46. When a destination bin iscomplete, the completed bin 51 is urged onto an output conveyor 48 forfurther processing or transport. The conveyor 34 may be motioncontrolled so that both the speed and the direction of the conveyor(e.g., rollers or belt) may be controlled. In certain embodiments, theconveyors 48 may be gravity biased to cause any destination container 46on a conveyor 48 to be delivered to an output end without active controlof the conveyors 48. Sensors 33 may track each of the supply bins 32 asthey travel along the conveyor 34, and the system may thereby know whereeach supply bin 32 is located at all times.

The inner conveyors 49 (also shown in FIG. 15) on which the destinationcontainers 46 ride, may also be gravity biased sloping downward in theopposite direction (with the lower end biased toward the processingsection 36). In further embodiments, one or both sets of conveyors 48,49 may be biased in either direction. The biasing of the destinationcontainers 46 ensures that the destination containers 46 remain next toeach other, and when a container 51 is removed, the remaining containers46 are brought together. Sensors 53, for example, may track each of thecontainers 46 as they are loaded onto the conveyors 49, and theprocessing system 70 may thereby know where each container 46 islocated. The biasing of the conveyors 48 ensures that completed bins 51are removed from the destination section 44.

With reference to FIG. 3, the processing section 36 also includes aperception unit 50 that looks down onto a conveyor that receives aselected supply bin 32. The processing station may include a conveyorsystem 37 that circulates supply bins 32 from and back to the inputconveyor 34 using the diverter conveyor 35. An end effector 41 of theprogrammable motion device 40 is programmed to grasp an object from thea supply bin 32, and move the object to deliver it to a desireddestination bin 46 by placing or dropping the object into the carriage42 that then shuttles the object to (and drops the object into) theselected destination container 46. The supply bin 32 may then bereturned to the input conveyor 34, and, optionally, brought to a furtherprocessing station. At the processing station 36 therefore, one or morevendor supply bins 32 are routed to an input area, and the programmablemotion device 40 is actuated to grasp an object from a bin 32, and toplace the object into a carriage 42. The processed vendor bins are thenreturned to the common input stream, and the carriage 42 that receivedthe object is moved away from the processing station 36 toward one of aplurality of destination bins 46.

The system 30 may also include one or more perception units 33 locatedon or near the input conveyor 34 for identifying indicia on an exteriorof each of the bins 32, providing perception data from which thecontents of the bin may be identified, and then knowing its relativeposition on the conveyor 34, track its location. It is assumed that thebins of objects are marked in one or more places on their exterior witha visually distinctive mark such as a barcode (e.g., providing a UPCcode) or radio-frequency identification (RFID) tag or mailing label sothat they may be sufficiently identified with a scanner for processing.The type of marking depends on the type of scanning system used, but mayinclude 1D or 2D code symbologies. Multiple symbologies or labelingapproaches may be employed. The types of scanners employed are assumedto be compatible with the marking approach. The marking, e.g. bybarcode, RFID tag, mailing label or other means, encodes a identifyingindicia (e.g., a symbol string), which is typically a string of lettersand/or numbers. The symbol string uniquely associates the vendor binwith a specific set of homogenous objects.

The operations of the system described above are coordinated with acentral processing system 70 as shown in FIG. 2 that communicates (e.g.,wirelessly) with the articulated arm 40, the perception units 33, 50 and53, as well as in-feed conveyor 34. This system determines from symbolstrings the UPC associated with a vendor bin, as well as the outbounddestination for each object. The central control system 70 is comprisedof one or more workstations or central processing units (CPUs). Forexample, the correspondence between UPCs or mailing labels, and outbounddestinations is maintained by a central control system in a databasecalled a manifest. The central control system maintains the manifest bycommunicating with a warehouse management system (WMS). The manifestprovides the outbound destination for each in-bound object.

In accordance with another embodiment of the invention, a system a 30′may include input conveyor 34 on which supply bins 32 (e.g., decantedvendor bins) are provided, a processing section 36′ that includes aprogrammable motion device 40, and a destination section 44 thatincludes one or more shuttle carriers 42. The programmable motion device40 is able to service two carriages 42 and two sets of destination bins46. Again, supply bins 32 are selectively provided to and from theprocessing section 36′ by a diverter conveyor 35 that moves bins ofobjects to the processing section 36. The programmable motion device 40then moves objects from a selected supply bin 32 and places or dropsthem into one of the two carriage 42 of the destination section fordelivery to one of a plurality of destination containers 46. When adestination bin is complete, the completed bin 51 is urged onto anoutput conveyor 48 for further processing or transport. The conveyor 34may be motion controlled so that both the speed and the direction of theconveyor (e.g., rollers or belt) may be controlled. In certainembodiments, the conveyors 48 may be gravity biased to cause anydestination container 46 on a conveyor 48 to be delivered to an outputend without active control of the conveyors 48. The sensors 33 may trackeach of the supply bins 32 as they are passed on the conveyor 34, andthe system may thereby know where each supply bin 32 is located at alltimes.

Again, the inner conveyors 49 on which the destination containers 46ride, may also be gravity biased sloping downward in the oppositedirection (with the lower end biased toward the processing section 36).In further embodiments, one or both sets of conveyors 48, 49 may bebiased in either direction. The biasing of the completed conveyors 51ensures that the destination containers 46 remain next to each other,and when a container 49 is removed, the remaining containers 46 arebrought together. The sensors 53, for example, may track each of thecontainers 46 as they are loaded onto the conveyors 51, and theprocessing system 70 may thereby know where each container 46 islocated. The biasing of the conveyors 48 ensures that completed bins 51are removed from the destination section 44.

With reference to FIG. 4, a system 30′ in accordance with anotherembodiment of the invention may include a processing section 36′ thatincludes a robot 40, which retrieves objects from an input bin 32 on aninput area conveyor 37′, and provides the objects to one of twocarriages 42. As discussed above, each of the carriages runs along atrack, bring the object among destination bins 46, and may be actuatedto drop the object into a selected destination bin 46. The remainingelements of the system 30′ are the same as those of the system 30.

The system of various embodiments includes a perception system (e.g.,50) that is mounted above a bin of objects to be processed next to thebase of the articulated arm 40, looking down into a bin 32. Theperception system 50, for example and as shown in FIG. 5, may include(on the underside thereof), a camera 72, a depth sensor 74 and lights76. A combination of 2D and 3D (depth) data is acquired. The depthsensor 74 may provide depth information that may be used together withthe camera image data to determine depth information regarding thevarious objects in view. The lights 76 may be used to remove shadows andto facilitate the identification of edges of objects, and may be all onduring use, or may be illuminated in accordance with a desired sequenceto assist in object identification. The system uses this imagery and avariety of algorithms to generate a set of candidate grasp locations forthe objects in the bin as discussed in more detail below.

FIG. 6 shows a view of the bin 32 from the perception unit 50. The imageview shows the bin 32 (e.g., on the conveyor), and the bin 32 containsobjects 78, 80, 82, 84 and 86. In the present embodiment, the objectsare homogenous, and are intended for distribution to differentdistribution packages. Superimposed on the objects 78, 80, 82, 84, 86(for illustrative purposes) are anticipated grasp locations 79, 81, 83and 85 of the objects. Note that while candidate grasp locations 79, 83and 85 appear to be good grasp locations, grasp location 81 does notbecause its associated object is at least partially underneath anotherobject. The system may also not even try to yet identify a grasplocation for the object 84 because the object 84 is too obscured byother objects. Candidate grasp locations may be indicated using a 3Dmodel of the robot end effector placed in the location where the actualend effector would go to use as a grasp location. Grasp locations may beconsidered good, for example, if they are close to the center of mass ofthe object to provide greater stability during grasp and transport,and/or if they avoid places on an object such as caps, seams etc. wherea good vacuum seal might not be available.

If an object cannot be fully perceived by the detection system, theperception system considers the object to be two different objects, andmay propose more than one candidate grasps of such two differentobjects. If the system executes a grasp at either of these bad grasplocations, it will either fail to acquire the object due to a bad grasppoint where a vacuum seal will not occur, or will acquire the object ata grasp location that is very far from the center of mass of the objectand thereby induce a great deal of instability during any attemptedtransport. Each of these results is undesirable.

If a bad grasp location is experienced, the system may remember thatlocation for the associated object. By identifying good and bad grasplocations, a correlation is established between features in the 2D/3Dimages and the idea of good or bad grasp locations. Using this data andthese correlations as input to machine learning algorithms, the systemmay eventually learn, for each image presented to it, where to bestgrasp an object, and where to avoid grasping an object.

As shown in FIGS. 7A and 7B, the perception system may also identifyportions of an object that are the most flat in the generation of goodgrasp location information. In particular, if an object includes atubular end and a flat end such as object 87, the system would identifythe more flat end as shown at 88 in FIG. 7B. Additionally, the systemmay select the area of an object where a UPC code appears, as such codesare often printed on a relatively flat portion of the object tofacilitate scanning of the barcode.

FIGS. 8A and 8B show that for each object 90, 92, the grasp selectionsystem may determine a direction that is normal to the selected flatportion of the object 90, 92. As shown in FIGS. 9A and 9B, the systemwill then direct the end effector 94 to approach each object 90, 92 fromthe direction that is normal to the surface in order to betterfacilitate the generation of a good grasp on each object. By approachingeach object from a direction that is substantially normal to a surfaceof the object, the robotic system significantly improves the likelihoodof obtaining a good grasp of the object, particularly when a vacuum endeffector is employed.

The invention provides therefore in certain embodiments that graspoptimization may be based on determination of surface normal, i.e.,moving the end effector to be normal to the perceived surface of theobject (as opposed to vertical picks), and that such grasp points may bechosen using fiducial features as grasp points, such as picking on abarcode, given that barcodes are almost always applied to a flat spot onthe object.

In accordance with various embodiments therefore, the invention furtherprovides a processing system that may learn object grasp locations fromexperience (and optionally human guidance). Systems designed to work inthe same environments as human workers will face an enormous variety ofobjects, poses, etc. This enormous variety almost ensures that therobotic system will encounter some configuration of object(s) that itcannot handle optimally; at such times, it is desirable to enable ahuman operator to assist the system and have the system learn fromnon-optimal grasps.

The system optimizes grasp points based on a wide range of features,either extracted offline or online, tailored to the gripper'scharacteristics. The properties of the suction cup influence itsadaptability to the underlying surface, hence an optimal grasp is morelikely to be achieved when picking on the estimated surface normal of anobject rather than performing vertical gantry picks common to currentindustrial applications.

In addition to geometric information the system uses appearance basedfeatures as depth sensors may not always be accurate enough to providesufficient information about graspability. For example, the system canlearn the location of fiducials such as barcodes on the object, whichcan be used as indicator for a surface patch that is flat andimpermeable, hence suitable for a suction cup. One such example is theuse of barcodes on consumer products. Another example is shipping boxesand bags, which tend to have the shipping label at the object's centerof mass and provide an impermeable surface, as opposed to the raw bagmaterial, which might be slightly porous and hence not present a goodgrasp.

By identifying bad or good grasp points on the image, a correlation isestablished between features in the 2D/3D imagery and the idea of goodor bad grasp points; using this data and these correlations as input tomachine learning algorithms, the system can eventually learn, for eachimage presented to it, where to grasp and where to avoid.

This information is added to experience based data the system collectswith every pick attempt, successful or not. Over time the robot learnsto avoid features that result in unsuccessful grasps, either specific toan object type or to a surface/material type. For example, the robot mayprefer to avoid picks on shrink wrap, no matter which object it isapplied to, but may only prefer to place the grasp near fiducials oncertain object types such as shipping bags.

This learning can be accelerated by off-line generation ofhuman-corrected images. For instance, a human could be presented withthousands of images from previous system operation and manually annotategood and bad grasp points on each one. This would generate a largeamount of data that could also be input into the machine learningalgorithms to enhance the speed and efficacy of the system learning.

In addition to experience based or human expert based training data, alarge set of labeled training data can be generated based on a detailedobject model in physics simulation making use of known gripper andobject characteristics. This allows fast and dense generation ofgraspability data over a large set of objects, as this process is notlimited by the speed of the physical robotic system or human input.

The system of an embodiment may also employ motion planning using atrajectory database that is dynamically updated over time, and isindexed by customer metrics. The problem domains contain a mix ofchanging and unchanging components in the environment. For example, theobjects that are presented to the system are often presented in randomconfigurations, but the target locations into which the objects are tobe placed are often fixed and do not change over the entire operation.

One use of the trajectory database is to exploit the unchanging parts ofthe environment by pre-computing and saving into a database trajectoriesthat efficiently and robustly move the system through these spaces.Another use of the trajectory database is to constantly improve theperformance of the system over the lifetime of its operation. Thedatabase communicates with a planning server that is continuouslyplanning trajectories from the various starts to the various goals, tohave a large and varied set of trajectories for achieving any particulartask. In various embodiments, a trajectory path may include any numberof changing and unchanging portions that, when combined, provide anoptimal trajectory path in an efficient amount of time.

FIG. 10 for example, shows a diagrammatic view of a system in accordancewith an embodiment of the invention that includes an input area conveyor37 that provide input bins 32 to a programmable motion device (as showndiagrammatically at 40), such as an articulated arm, having a base asshown at 110, and an end effector (shown diagrammatically at 41) that isprogrammed to have a home position (shown at 112), and is programmed formoving objects from an input bin 32 to processing locations, e.g.,destination locations at the plurality of destination locations 46.Again, the system may include a defined home or base location 110 towhich each object may initially be brought upon acquisition from the bin32.

In certain embodiments, the system may include a plurality of baselocations, as well as a plurality of predetermined path portionsassociated with the plurality of base locations. The trajectories takenby the articulated arm of the robot system from the input bin to thebase location are constantly changing based in part, on the location ofeach object in the input bin, the orientation of the object in the inputbin, and the shape, weight and other physical properties of the objectto be acquired.

Once the articulated arm has acquired an object and is positioned at thebase location, the paths to each of the plurality of destinationcarriers 46 are not changing. In particular, each destination bin isassociated with a unique destination bin location, and the trajectoriesfrom the base location to each of the destination bin locationsindividually is not changing. A trajectory, for example, may be aspecification for the motion of a programmable motion device over time.In accordance with various embodiments, such trajectories may begenerated by experience, by a person training the system, and/or byautomated algorithms. For a trajectory that is not changing, theshortest distance is a direct path to the target destination bin, butthe articulated arm is comprised of articulated sections, joints, motorsetc. that provide specific ranges of motion, speeds, accelerations anddecelerations. Because of this, the robotic system may take any of avariety of trajectories between, for example, base locations anddestination bin locations.

FIG. 11 for example, shows three such trajectories (₁T¹, ₂T¹ and ₃T¹)between base location 112 and a carriage location at 42. The elements ofFIG. 11 are the same as those of FIG. 10. Each trajectory will have anassociated time as well as an associated risk factor. The time is thetime it takes for the articulated arm of the robotic system toaccelerate from the base location 112 move toward the carriage bin 42,and decelerate to the carriage bin 42 in order to place the object inthe carriage bin 42.

The risk factor may be determined in a number of ways including whetherthe trajectory includes a high (as pre-defined) acceleration ordeceleration (linear or angular) at any point during the trajectory. Therisk factor may also include any likelihood that the articulated arm mayencounter (crash into) anything in the robotic environment. Further, therisk factor may also be defined based on learned knowledge informationfrom experience of the same type of robotic arms in other roboticsystems moving the same object from a base location to the samedestination location.

As shown in the table at 96 in FIG. 11, the trajectory ₁T¹ from the baselocation 112 to the carriage location 42 may have a fast time (0.6 s)but a high risk factor. The trajectory ₂T¹ from the base location 112 tothe carriage location 42 may have a much slower time (1.4 s) but still afairly high risk factor (16.7). The trajectory ₃T¹ from the baselocation 112 to the carriage location 42 may have a relatively fast time(1.3 s) and a moderate risk factor (11.2). The choice of selecting thefastest trajectory is not always the best as sometimes the fastesttrajectory may have an unacceptably high risk factor. If the risk factoris too high, valuable time may be lost by failure of the robotic systemto maintain acquisition of the object. Different trajectories therefore,may have different times and risk factors, and this data may be used bythe system in motion planning.

FIG. 12, for example, shows minimum time-selected trajectories from thebase location 112 to the carriage locations 42. In particular, thetables shown at 97 that the time and risk factors for a plurality ofpaths, and the trajectories from the base location 112 to the carriagelocation 42 are chosen to provide the minimum time for motion planningfor motion planning under a risk factor of 14.0.

FIG. 13 shows minimum risk-factor-selected set of trajectories from thebase location 112 to the carriage location 42. Again, the tables shownat 97 show the time and risk factors for the various paths. Thetrajectories from the base location 112 to each of the carriage 42 arechosen to provide the minimum risk factor for motion planning for motionplanning under a maximum time of 1.2 seconds.

The choice of fast time vs. low risk factor may be determined in avariety of ways, for example, by choosing the fastest time having a riskfactor below an upper risk factor limit (e.g., 12 or 14), or by choosinga lowest risk factor having a maximum time below an upper limit (e.g.,1.0 or 1.2). Again, if the risk factor is too high, valuable time may belost by failure of the robotic system to maintain acquisition of theobject. An advantage of the varied set is robustness to small changes inthe environment and to different-sized objects the system might behandling: instead of re-planning in these situations, the systemiterates through the database until it finds a trajectory that iscollision-free, safe and robust for the new situation. The system maytherefore generalize across a variety of environments without having tore-plan the motions.

Overall trajectories therefore, may include any number of changing andunchanging sections. For example, networks of unchanging trajectoryportions may be employed as commonly used paths (roads), while changingportions may be directed to moving objects to a close-by unchangingportion (close road) to facilitate moving the object without requiringthe entire route to be planned. For example, the programmable motiondevice (e.g., a robot) may be tasked with orienting the grasped objectin front of an automatic labeler before moving towards the destination.The trajectory to sort the object therefore, would be made up of thefollowing trajectory portions: First, a grasp pose to a home position(motion planned). Then, from home position to an auto-labeler home(pulled from a trajectory database). Then, from the auto-labeler home toa labelling pose (motion planned). Then, from the labelling pose to anauto-labeler home (either motion planned or just reverse the previousmotion plan step). Then, from the auto-labeler home to the intendeddestination (pulled from the trajectory database). A wide variety ofchanging and unchanging (planned and pulled from a database) portionsmay be employed in overall trajectories. In accordance with furtherembodiments, the object may be grasped from a specific pose (planned),and when the object reaches a destination bin (from the trajectorydatabase), the last step may be to again place the object in the desiredpose (planned) within the destination bin.

In accordance with further embodiments, the motion planning may alsoprovide that relatively heavy items (as may be determined by knowinginformation about the grasped object or by sensing weight—or both—at theend effector) may be processed (e.g., moved in trajectories) and placedin boxes in very different ways than the processing and placement ofrelatively light objects. Again, the risk verses speed calculations maybe employed for optimization of moving known objects of a variety ofweights and sizes as may occur, for example, in the processing of a widevariety of consumer products. The system, therefore, provides means thatinterface with the customer's outgoing object conveyance systems. When abin (or package) is full as determined by the system (in monitoringsystem operation), a human operator may pull the bin from the processingarea, and place the bin in an appropriate conveyor. When a bin is fulland gets removed to the closed/labelled location, another empty bin isimmediately placed in the location freed up by the removed full bin, andthe system continues processing as discussed above.

FIG. 14 shows the destination section 44 of the system 30 that includesa movable carriage 42 that may receive an object 54 from the endeffector of the programmable motion device. The movable carriage 42 isreciprocally movable between two rows of the destination bins 46 along aguide rail 45. As shown in FIG. 14, each destination bin 46 includes aguide chute 47 that guides an object dropped therein into the underlyingdestination bin 46. The carriage 42 moves along a track 45 (as furthershown in FIG. 15), and the carriage may be actuated to drop an object 54into a desired destination bin 46 via a guide chute 47 (as shown in FIG.16).

The movable carriage 42 is therefore reciprocally movable between thedestination bins, and the/each carriage moves along a track, and may beactuated to drop an object into a desired destination bin 24. Thedestination bins may be provided in a conveyor (e.g., rollers or belt),and may be biased (for example by gravity) to urge all destination binstoward one end (for example, the distal end. When a destination bin isselected for removal (e.g., because the bin is full or otherwise readyfor further processing), the system will urge the completed bin onto anoutput conveyor to be brought to a further processing or shipmentstation. The conveyor may be biased (e.g., by gravity) or powered tocause any bin on the conveyor to be brought to an output location.

FIGS. 17A and 17B show a bin 51 being urged from the plurality ofdestination bins 46, onto the output conveyor 48 by the use of adisplacement mechanism 55. In accordance with further embodiments, thedestination bins may be provided as boxes or containers or any othertype of device that may receive and hold an item, including box trayassemblies as discussed below.

Following displacement of the bin onto the conveyor (as shown in FIG.18), each of the destination bins may be urged together (as shown inFIG. 19) and the system will record the change in position of any of thebins that moved. This way, a new empty bin may be added to the end, andthe system will record the correct location and identified processingparticulars of each of the destination bins.

As noted above, the bins 46 may be provided as boxes, totes, containersor any other type of device that may receive and hold an item. Infurther embodiments, the bins may be provided in uniform trays (toprovide consistency of spacing and processing) and may further includeopen covers that may maintain the bin in an open position, and mayfurther provide consistency in processing through any of spacing,alignment, or labeling.

For example, FIG. 20 shows an exploded view of a box tray assembly 130.As shown, the box 132 (e.g., a standard shipping sized cardboard box)may include bottom 131 and side edges 133 that are received by a topsurface 135 and inner sides 137 of a box tray 134. The box tray 134 mayinclude a recessed (protected) area in which a label or otheridentifying indicia 146 may be provided, as well as a wide and smoothcontact surface 151 that may be engaged by an urging or removalmechanism as discussed below.

As also shown in FIG. 20, the box 132 may include top flaps 138 that,when opened as shown, are held open by inner surfaces 140 of the boxcover 136. The box cover 136 may also include a recessed (protected)area in which a label or other identifying indicia 145 may be providedThe box cover 136 also provides a defined rim opening 142, as well ascorner elements 144 that may assist in providing structural integrity ofthe assembly, and may assist in stacking un-used covers on one another.Un-used box trays may also be stacked on each other.

The box 132 is thus maintained securely within the box tray 134, and thebox cover 136 provides that the flaps 138 remain down along the outsideof the box permitting the interior of the box to be accessible throughthe opening 142 in the box cover 136. FIG. 21 shows a width side view ofthe box tray assembly 130 with the box 132 securely seated within thebox tray 134, and the box cover holding open the flaps 138 of the box132. The box tray assemblies may be used as any or both of the storagebins and destination bins in various embodiments of the presentinvention.

With reference to FIGS. 22A-22D, a box kicker 184 in accordance with anembodiment of the present invention may be suspended by and travel alonga track 86, and may include a rotatable arm 88 and a roller wheel 190 atthe end of the arm 88. With reference to FIGS. 22B-22D, when the rollerwheel 190 contacts the kicker plate 151 (shown in FIG. 20) of a box trayassembly 120, the arm 188 continues to rotate, urging the box trayassembly 180 from a first conveyor 182 to a second conveyor 180. Again,the roller wheel 190 is designed to contact the kicker plate 151 of abox tray assembly 181 to push the box tray assembly 181 onto theconveyor 180. Such a system may be used to provide that boxes that areempty or finished being unloaded may be removed (e.g., from conveyor182), or that boxes that are full or finished being loaded may beremoved (e.g., from conveyor 182). The conveyors 180, 182 may also becoplanar, and the system may further include transition roller 183 tofacilitate movement of the box tray assembly 181, e.g., by beingactivated to pull the box tray over to the conveyor 180.

Systems of the invention are highly scalable in terms of sorts-per-houras well as the number of storage bins and destination bins that may beavailable. FIG. 23 shows a system 200 in accordance with a furtherembodiment of the present invention that includes plurality ofprocessing sections 202 that access a common input conveyor 204, and aplurality of destination sections 206 for receiving objects via ashuttle carrier 208. Generally, each processing section 202 includes aprogrammable motion device 210 that provides objects to a shuttlecarrier 208 of one or more processing sections. Supply bins 214 areprovided on the input conveyor 204, and destination bins 216 areprovided at the destination sections 206 and are accessible by a shuttlecarrier 208.

FIG. 24 shows a system 300 in accordance with a further embodiment ofthe present invention that includes a further plurality of many rows ofsets of processing sections 302 that access one of a plurality of inputconveyors 304, and process objects to be placed in any of a variety ofdestination bins via a plurality of programmable motion devices and aplurality of shuttle carriers that bring objects to desired destinationsthat are then provided to an output conveyor 348.

Control of each of the systems 30, 30′, 200 and 300 may be provided bythe computer system 70 that is in communication with the storageconveyors and displacement mechanism(s), the processing conveyors anddisplacement mechanism(s), and the programmable motion device(s). Thecomputer system 70 also contains the knowledge (continuously updated) ofthe location and identity of each of the storage bins, and contains theknowledge (also continuously updated) of the location and identity ofeach of the destination bins. The system therefore, directs the movementof the storage bins and the destination bins, and retrieves objects fromthe storage bins, and distributes the objects to the destination bins inaccordance with an overall manifest that dictates which objects must beprovided in which destination boxes for shipment, for example, todistribution or retail locations.

Those skilled in the art will appreciate that numerous modifications andvariations may be made to the above disclosed embodiments withoutdeparting from the spirit and scope of the present invention.

What is claimed is:
 1. A processing system for processing objects usinga programmable motion device, said processing system comprising: aplurality of supply bins providing supply of a plurality of objects,said plurality of supply bins being provided with a bin conveyancesystem; a programmable motion device in communication with the binconveyance system, said programmable motion device including an endeffector for grasping and moving a selected object out of a selectedsupply bin; a plurality of destination containers provided on an infeedconveyor, wherein the infeed conveyor is sloped to gravity bias theplurality of destination containers toward the programmable motiondevice; and a movable carriage for receiving the selected object fromthe end effector of the programmable motion device, and for carrying theselected object to one of the plurality of destination containers on theinfeed conveyor.
 2. The processing system as claimed in claim 1, whereinthe movable carriage reciprocally moves between two rows of infeedconveyors holding the plurality of destination containers.
 3. Theprocessing system as claimed in claim 1, wherein the movable carriage isadapted to drop the selected object into a selected destinationcontainer.
 4. The processing system as claimed in claim 1, wherein thedestination containers are provided adjacent to an output conveyor. 5.The processing system as claimed in claim 1, wherein the processingsystem further includes at least one further programmable motion devicefor grasping and moving a further selected object out of a furtherselected storage bin, and for providing the further selected object to afurther movable carriage.
 6. The processing system as claimed in claim1, further comprising an infeed sensor that senses an identity of adestination container moving past the infeed sensor in order to index alocation of each destination container on the infeed conveyor.
 7. Theprocessing system as claimed in claim 1, wherein the infeed conveyorterminates adjacent the programmable motion device.
 8. The processingsystem as claimed in claim 1, further comprising a carriage rail betweenparallel rows of the destination containers, wherein a single movablecarriage is positioned on the rail.
 9. The processing system as claimedin claim 1, wherein the bin conveyance system includes a diverter forselectively providing a supply bin to the programmable motion device.10. The processing system as claimed in claim 9, wherein theprogrammable motion device includes an articulated arm, and wherein thearticulated arm is positioned adjacent to the diverter.
 11. Theprocessing system as claimed in claim 1, wherein the processing systemfurther includes a container displacement system for displacing eachdestination container onto an output conveyor.
 12. The processing systemas claimed in claim 11, wherein the output conveyor is gravity biased toprovide a destination container thereon to a further processinglocation.
 13. The processing system as claimed in claim 12, wherein thefurther processing location is a shipment transport location.
 14. Theprocessing system as claimed in claim 11, wherein one or more of theplurality of destination containers positioned higher on the slopedinfeed conveyor are gravity biased to move toward the programmablemotion device to fill a space left by the displaced destinationcontainer.
 15. The processing system as claimed in claim 14, wherein anew destination container is provided at the top of the sloped infeedconveyor to move down the sloped infeed conveyor and refill thedestination containers.
 16. The processing system as claimed in claim14, further comprising an infeed sensor that senses an identity of eachdestination container moving past the infeed sensor in order to index alocation of each destination container on the infeed conveyor.
 17. Aprocessing system for processing objects using a programmable motiondevice, said processing system comprising: a programmable motion deviceincluding an end effector for grasping and moving a selected object froma plurality of objects; a movable carriage for receiving the selectedobject from the end effector of the programmable motion device, and forcarrying the selected object to one of a plurality of destinationcontainers; and a destination container displacement system for urging aselected destination container onto an output conveyor; wherein themovable carriage reciprocally moves between two parallel rows of thedestination containers, and wherein the movable carriage pivots towardsone of the two rows of destination containers to drop the selectedobject into a selected destination containers.
 18. The processing systemas claimed in claim 17, wherein the two rows of destination containerare provided adjacent to the output conveyor.
 19. The processing systemas claimed in claim 17, wherein the output conveyor is gravity biased toprovide a destination container thereon to a further processinglocation.
 20. The processing system as claimed in claim 17, wherein theprocessing system further includes at least one further programmablemotion device for grasping and moving a further selected object from theplurality of objects, and for providing the further selected object to afurther movable carriage.
 21. The processing system as claimed in claim17, wherein the system further includes a plurality of supply binsproviding supply of the plurality of objects.
 22. The processing systemas claimed in claim 21, wherein the plurality of supply bins areprovided with a bin conveyance system.
 23. The processing system asclaimed in claim 22, wherein the bin conveyance system includes adiverter for selectively providing a supply bin to the programmablemotion device.
 24. The processing system as claimed in claim 23, whereinthe programmable motion device includes an articulated arm, and whereinthe articulated arm is positioned adjacent to the diverter.
 25. A methodof processing of objects, said method comprising the steps of: providinga plurality of supply bins including a plurality of objects, saidplurality of supply bins being provided with a bin conveyance system;providing a plurality of destination containers on a pair of infeedconveyors, each infeed conveyor being adjacent an outfeed conveyor;grasping and moving a selected object out of a selected supply bin usinga programmable motion device; providing the selected object to areciprocating carriage; carrying the selected object to a selected oneof the plurality of destination containers using the reciprocatingcarriage; depositing the selected object from the reciprocating carriageto the selected destination container; displacing the selecteddestination container onto an output conveyor with a displacementdevice; and collapsing the destination containers that are upstream tothe displaced destination carrier in the downstream direction to fill aspace left by the selected displaced destination container.
 26. Themethod as claimed in claim 25, wherein the method further includes thestep of selectively diverting a supply bin to the programmable motiondevice.
 27. The method as claimed in claim 25, wherein the reciprocatingcarriage reciprocally moves between two rows of the destination bins.28. The method as claimed in claim 25, wherein the reciprocatingcarriage is adapted to drop objects, and the step of depositing includesdropping the selected object into the selected destination container.29. The method as claimed in claim 25, wherein each infeed conveyor issloped toward a programmable motion device, and the step of collapsingincludes gravity biasing the destination containers toward theprogrammable motion device.
 30. The method as claimed in claim 25,wherein the output conveyor is gravity biased to provide a destinationcontainer thereon to a further processing location.
 31. The method asclaimed in claim 30, wherein the further processing location is ashipment transport location.
 32. The method as claimed in claim 25,wherein the step of displacing each destination container onto an outputconveyor includes actuating a rotatable box kicker to engage and push adestination container onto the output conveyor.
 33. The method asclaimed in claim 32, wherein the box kicker is mounted on a railpositioned between the parallel infeed conveyors.